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APPARATUS FOR AND METHOD OF VARIABLE BIT RATE VIDEO CODING 
BACKGROUND OF THE INVENTION 

The present invention relates to an apparatus for 
and a method of variable bit rate video coding and, more 
5 particularly, to those in which the generated code bit 
count is controlled in real time at a variable bit rate, 
MPEG-2 ( ISO/IEC-13818-2 ) , for instance, is well 
known in the art as a method of high efficiency video 
coding. In this method, the picture is divided into a 

10 plurality of blocks each constituted by a group of a 
plurality of pixels, and a special domain signal is 
converted to a frequency domain signal by executing 
discrete cosine transform (DCT) on each block. Each 
frequency component called DCT coefficient obtained by 

15 the discrete cosine transform, is quantized with a 

quantization step size, which is predetermined for each 
coefficient position (i.e., frequency). The quantized 
data is variable length coded by assigning variable 
length codes to the DCT coefficients. The coded data thus 

20 obtained is provided as a bit stream. 

Fig. 8 is a view for describing an image coding 
method conforming to the MPEG-2 ( ISO/IEC-13818-2 ) . 
Referring to the Figure, the method is implemented by 
a system comprising a subtracter 501, a discrete cosine 

25 transform unit 502, a quantizer 503, a variable length 
coder 504 , a bit rate controller 505 , an inverse quantizer 
506, an inverse discrete cosine transform unit 507, an 
adder 508, a frame memory 509, a motion compensated 
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inter-frame predictor 510 and a selector 511. 

Input picture images are classified into I pictures 
to be intra-frame coded, P pictures to be inter-frame 
prediction coded by using only forward prediction and 
5 B pictures, for which prediction in both forward and 
backward is made. The input image is divided into a 
plurality of macro-blocks each of 16 x 16 pixels for 
coding the image in units of macro-blocks. 

I pictures are intra-frame coded without the 

10 inter-frame prediction. In this case, the selector 511 
selects signal value of "0" as prediction value, and the 
same value is outputted as input image signal value from 
the subtracter 501. The discrete cosine transform unit 
502 executes discrete cosine transform on the output 

15 signal value from the subtracter 501, and thus outputs 
a DCT coefficient. The quantizer 503 quantizes the DCT 
coefficient with a predetermined quantization step size. 

The quantizer 503 outputs a quantized DCT 
coefficient as quantized coefficient to the variable 

20 length coder 504. The variable length coder 504 encodes 
the quantized coefficient with variable length codes. 
The variable length coded quantized coefficient is 
outputted together with other data as a bit stream. The 
inverse quantizer 506 inversely quantizes the quantized 

25 coefficient output of the quantizer 503. The inverse 
discrete cosine transform unit 507 executes inverse 
discrete cosine transform on the inversely quantized 
coefficient output of the inverse quantizer 503 thus 
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reconstructing the original input image signal* 

The reconstructed image signal that is obtained 
through the inverse quantizer 506 and the inverse 
discrete cosine transform unit 507, is used as reference 
5 image in subsequently executed inter-frame prediction, 
and is stored through the adder 508 in the frame memory 
509. The motion compensation inter-frame predictor 510 
executes inter-frame prediction on the P and B pictures. 
The predictor 510 first determines a motion vector by 

10 comparing the input image signal and the reference image 
stored in the frame memory 509 and executing motion 
estimation for each macro-block as a division of the input 
image. Then, the predictor 501 determines a prediction 
mode based on the result of the motion estimation for 

15 each macro-block. In this process, it is determined 
whether it is better to or not to set an intra-frame mode, 
in which a macro-block coding process is executed on P 
and B pictures . 

When it is determined that it is better to set the 

20 intra-frame mode for coding process determined in a 
macro-block prediction mode, the macro-block coding 
process on P and B pictures is the same as the above coding 
process on I pictures, that is, intra-frame coding 
process is executed without motion compensated 

25 inter-frame prediction according to any determined 
motion vector. 

When it is determined that it is better not to set 
the intra-frame mode, that is, it is better to set the 
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non- intra- frame mode in the macro-block prediction mode, 
the macro-block coding process on P and B pictures is 
executed with infra-frame prediction by using the image 
signal recorded in the frame memory 509 as reference image. 
5 The motion compensated inter-frame predictor 510 
generates a predicted image signal corresponding to 
input image signal on the basis of a previously determined 
motion vector. 

The predicted image signal thus generated is 

10 provided to the subtracter 510 and also to the adder 508. 
In the following, the predicted image signal supplied 
to the subtracter will be described. Receiving the 
predicted image signal, the subtracter 501 computes the 
differential signal between the input image signal and 

15 the predicted image signal. The subsequent coding 

process on the differential signal is the same as the 
coding process on I pictures . That is , the discrete cosine 
transform unit 502 generates DCT coefficients from the 
differential signal, and the quantizer 503 quantizes the 

20 DCT coefficients. 

The differential signal outputted from the 
quantizer 503 is provided to the variable length coder 
504 for providing a bit stream, and it is also provided 
to the inverse quantizer 506. Through the inverse 

25 quantizer 506 and the inverse cosine transform unit 507, 
a reconstructing process is executed on the coded 
differential signal. This reconstructing process is the 
same as the reconstructing process on P and B pictures 
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as described above. Specifically, the adder 502 adds the 
reconstructed differential signal and the predicted 
image signal previously provided thereto, and thus a 
reconstructed image signal is generated. The 
5 reconstructed image signal generated in the adder 508 
is stored in the frame memory 509 to be used as reference 
image in a subsequent coding process by inter-frame 
prediction. 

As shown above, in a method represented by 

10 MPEG-2(ISO/lEC-1388-2) , for instance, an efficient 

coding process can be executed on the input video signal 
since it is possible to reduce the redundancy in the 
spatial domain in DCT and also reduce the time domain 
redundancy in the inter-frame prediction. In this method, 

15 the bit rate controller 505 which is provided between 
the variable length coder 504 and the quantizer 503, 
obtains the generated bit count from the variable length 
coder 504, then determines the quantization step size 
such as to meet a bit rate restriction, and transmits 

20 the determined quantization step size data to the 

quantizer 503 for controlling the generated bit count. 

Prior art examples of coding process employed in 
the above method typically represented by the MPEG- 
2(ISOP/IEC-13818-2 will now be described. 

25 As a first prior art example, a bit rate control 

method in a test model (i.e., Test Model 5, ISO/ICE 
JTC1/SC21/WG11/N0400, April in 1993) of MPEG-2 is well 
known as a bit count control method in a coding method, 
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which involves the above quantizing process. 

This method adopts a constant bit rate coding 
method, which seeks to set a generated bit count in 
intra-frame coding process and coding process by 
5 inter-frame prediction to constant bit count for each 
of certain time units. Specifically, in this method the 
bit count is controlled by adjusting the quantization 
step size set for each macro-blocks obtained by dividing 
the picture frame into 16 X 16 pixels. 

10 In this first prior art example, since the constant 

bit rate control method is adopted, it is necessary to 
control the bit count per unit time to be constant. The 
generated bit count is reduced by increasing the 
quantization step size for scenes, which requires high 

15 bit count code generation, and increased by reducing the 
quantization step size for scenes, which do not require 
so high bit rate code generation. 

In this example, however, the quantization step 
size is set to control the bit count per unit time to 

20 be constant independently input images. Therefore, in 
a scene requiring a high bit count, the image quality 
is degraded, because the generated code bit count is 
suppressed by a large quantization step size. 

To replace the constant bit rate coding method of 

25 the above first prior art example, a variable bit rate 
coding method has been proposed. In an example of 
variable bit rate coding method, when an encoder records 
images on storage media which have a predetermined total 
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recording bit count, the output bit rate is controlled 
according to the bit count required for each image, thus 
improving the average image quality while meeting the 
restriction imposed on the recording capacity of the 
5 storage media. 

As a second prior art example of coding process, 
Japanese Patent Laid-Open No. 6-141298 discloses a 
process , which adopts the above reliable bit rate control 
method. In this second prior art example, a provisional 

10 coding process is first executed, in which DCT 

coefficient quantization is executed on the basis of a 
preliminarily set reference quantization step size for 
computing generated bit count, and then a regular coding 
process is executed for actual coding. This coding 

15 process is thus a "two-pass coding" system, in which the 
regular coding process is executed after completion of 
the provisional coding process. In other words, re- 
coding is executed after obtaining the knowledge of the 
characteristics of the whole contents, and it is thus 

20 possible to attain high image quality coding. 

In this second prior art example, however, the 
coding of a certain video content requires execution of 
two processes, i.e., the provisional and regular coding 
processes, thus doubling the process time. Therefore, 

25 it is impossible to encode in real time processing. 

For real time execution of the variable bit rate 
coding, a "single path coding" system, which does not 
execute any provisional coding process but executes the 
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single regular coding process, has been considered. 

As a third prior art example of coding process, "An 
Algorithm of MPEG2 Real time Variable Bit Rate Video 
Coding with Quantizer Step" (Inada et al, Information 
5 System Journal 2 D-ll-3 (page 3), General Conference of 
the Institute of Electronics, Information and 
Communication Engineers in March, 1998), is well known 
as "single pass coding" system in the above variable bit 
rate coding system. 

10 In this third prior art example, an image coding 

process conforming to MPEG-2 is executed, and after a 
predetermined period of time the generated bit count is 
adjusted by setting a quantization step size for each 
GOP to provide a predetermined average bit rate. 

15 Japanese Patent Laid-Open No. 10-164577 discloses 

a fourth prior art example of coding process. In this 
fourth prior art example, the actual generated bit count 
is used to update the bit count which can be assigned 
to subsequent several GOPs, and a target bit count or 

20 a target quantization step size is determined for each 
GOP on the basis of the ratio of the coding complexity 
of the present image to the average coding complexity 
over past images. 

However, the above signal pass variable bit rate 

25 coding systems have a problem that a series of coding 
processes may result in deterioration of the image 
quality. This is so because in the single pass coding 
systems a bit count capable of being assigned or a target 
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bit rate is computed before computation of the target 
bit count or quantization step size. This means that 
preference is given to bit rate control in a fixed time 
period, resulting in insufficient quantization step size 
5 control depended on the image scene. Consequently, the 
coded image quality may not substantially differ in 
comparison to the constant bit rate case. 

In addition, in the case of that the bit count is 
less than the average bit rate although the image does 

10 not require high bit rate code generation in low 

complexity scenes, the generated code bit count is sought 
to be increased by reducing the quantization step size 
in spite of the fact that sufficient image quality is 
obtained. This may result in an excessive code assignment 

15 and wasteful code consumption. Consequently, it may 
become impossible to assign sufficient bit count with 
an image which requires more bits in high complexity 
scenes . 

S U MMAR Y OF THE IN VE NT ION 

20 The present invention has been made in view of the 

above problems, and it has an object of providing a single 
pass variable bit rate coding system, which permits real 
time processing while ensuring improved image quality 
compared to the image quality obtainable with the prior 

25 art single pass variable bit rate coding systems. 

According to the present invention, there is 
provided an apparatus for variable bit rate video coding 
on the basis of a predetermined average bit rate 
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comprising: a video coding means for coding input video 
with a predetermined quantization step size and 
providing coded data and a generated bit count; a 
quantization step size setting means for setting a 
5 reference quantization step size for each first image 
unit in correspondence to the average bit rate from the 
quantization step size provided to the video coding means 
and also the generated bit count provided therefrom; and 
a quantization step size adjusting means for adjusting 

10 the quantization step size provided from the 

quantization step size setting means for each second 
image unit from the generated bit count provided from 
the video coding means and also from an excess or a 
shortage of the average bit rate. 

15 The first image unit is a group of a plurality of 

picture frames . 

The second image unit is a group of a plurality of 
macro-blocks as divisions of one picture frame. 

The quantization step size setting means for 

20 setting the reference quantization step size for each 
first image unit computes the first image unit complexity 
defined by the product of the average quantization step 
size over the preceding coded image and generated bit 
count, and also computes the reference quantization step 

25 size for each first image unit from the ratio between 
the first image unit complexity and the average bit rate. 

The quantization step size adjusting means for 
adjusting the quantization step size for each second 
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image unit sets a minimum permissible quantization step 
size. 

The quantization step size adjusting means for 
adjusting the quantization step size for each second 
image unit sets a maximum excess of the average bit rate, 
sets a virtual buffer sized to the maximum excess from 
which codes are withdrawn at the average bit rate and 
adjusts the quantization step size for each second image 
unit by using the buffer occupancy in the virtual buffer 
and a utilizable buffer size obtained by subtracting the 
buffer occupancy in the virtual buffer from the size 
thereof • 

The quantization step size setting means for 
setting the reference quantization step size for each 
second image unit includes a means for computing a first 
quantization step size by adjusting the quantization 
step size for each second image unit from the generated 
bit count provided from the video coding means and the 
excess or shortage of the average bit rate; and it sets 
a maximum bit rate, computes a second quantization step 
size, which is set in the case of constant bit rate control 
on the basis of the maximum bit rate, from the 
quantization step size set in the quantization step size 
setting means and the generated bit count provided from 
the video coding means, and provides the greater one of 
the first and second quantization step sizes to the video 
coding means . 

The quantization step size setting means for 



11 



setting the reference quantization step size for each 
first image unit computes a first image unit complexity 
for each first image unit f the first image unit complexity 
being defined by the product of the summation of generated 
bit count and the average quantization step size for each 
first image unit, and determines the quantization step 
size of the first image unit from the first image unit 
complexity of the first image unit and the bit count per 
first image unit. 

The quantization step size setting means for 
setting the reference quantization step size for each 
first image unit computes the first image unit complexity 
for each first image unit from the summation of the 
generated bit count and the average quantization step 
size for each first image unit, and determines the 
quantization step size of the first image unit from the 
average first image unit complexity for the first image 
unit over the preceding coded image, the average first 
image unit complexity of the immediately preceding first 
image unit or a plurality of first image unit including 
the immediately preceding first image unit and the bit 
rate per first image unit. 

The quantization step size adjusting means for 
adjusting the quantization step size for each second 
image unit preliminarily sets a threshold for 
quantization step size, when the bit balance to the 
average bit rate is not excessive, the reference 
quantization step size set for each first image unit is 
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compared with the threshold for quantization step size, 
for providing the quantization step size without any 
adjustment when the reference quantization step size is 
not exceeding the threshold for quantization step size, 
and adjusting the quantization step size according to 
the bit balance to the average bit rate and selectively 
providing the greater one of the adjusted quantization 
step size and the threshold for quantization step size, 
and when the bit balance to the average bit rate is 
excessive, the quantization step size is adjusted 
according to the excess or shortage of the average bit 
rate, the adjusted quantization step size being provided 
as the quantization step size for each second image unit. 

According to the present invention, there is 
provided a method of variable bit rate video coding on 
the basis of a predetermined average bit rate comprising: 
a video coding step of coding input video with a 
predetermined quantization step size and providing coded 
data and a generated code bit count; a quantization step 
size setting step of setting a reference quantization 
step size for each first image unit in correspondence 
to the average bit rate from the quantization step size 
provided to the video coding step and also the generated 
code bit count provided therefrom; and a quantization 
step size adjusting means for adjusting the quantization 
step size provided from the quantization step size 
setting means for each second image unit from the 
generated code bit count provided from the video coding 
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step and also from an excess or a shortage of the average 
bit rate. 

The first image unit is a group of a plurality of 
picture frames . 
5 The second image is a group of a plurality of 

macro-blocks as divisions of one picture frame. 

In the quantization step size setting step of 
setting the reference quantization step size for each 
Q first image unit, a first image unit complexity is 

? 10 computed, which is defined by the product of the average 
1 1 quantization step size and generated code bit rate over 

{2 the preceding coded image, and also the reference 

- y quantization step size for each first image unit is 

^ computed from the ratio between the first image unit 

IV 15 complexity and the average bit rate. 

;J3 In the quantization step size adjusting step of 

" adjusting the quantization step size for each second 

image unit, a minimum permissible quantization step size 

is set. 

20 In the quantization step size adjusting step of 

adjusting the quantization step size for each second 
image unit, a maximum excess of the average bit rate is 
set, a virtual buffer sized to the maximum excess is set 
for withdrawing codes at the average code bit, and the 

25 quantization step size is adjusted for each second image 
unit by using the occupation content in the virtual buffer 
and a utilizable code bit count obtained by subtracting 
the occupation content in the virtual buffer from the 
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size thereof • 

The quantization step size setting step of setting 
the reference quantization step size for each second 
image unit includes a step of computing a first 
5 quantization step size by adjusting the quantization 
step size for each second image unit from the generated 
code bit count provided from the video coding step and 
the excess or shortage of the average bit rate; and in 
the computing step, a maximum bit rate is set, a second 

10 quantization step size is computed, which is set in the 
case of fixed bit rate control on the basis of the maximum 
bit rate, from the quantized width set in the quantization 
step size setting step and the generated code bit count 
provided from the video coding step, and the greater one 

15 of the first and second quantization step sizes is 
provided to the video coding step. 

In the quantization step size setting means for 
setting the reference quantization step size for each 
first image unit, a first image unit complexity is 

20 computed for each first image unit, the first image unit 
being defined by the product of the summation of generated 
code bit rate and the average quantization step size for 
each first image unit, and the quantization step size 
of the first image is determined from the first image 

25 unit complexity of the first image unit and the bit rate 
per first image unit. 

In the quantization step size setting means for 
setting the reference quantization step size for each 
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first image unit, the first image unit complexity is 
computed for each first image unit from the summation 
of the generated code bit count and the average 
quantization step size for each first image unit, and 
5 the quantization step size of the first image unit is 
determined from the average first image unit complexity 
for the first image unit over the preceding coded image, 
the average first image unit complexity of the 
immediately preceding first image unit or a plurality 

10 of first image unit including the immediately preceding 
first image unit and the bit rate per first image unit. 

in the quantization step size adjusting means of 
adjusting the quantization step size for each second 
image unit, a threshold quantization step size is 

15 preliminarily set, when the access or shortage of the 
average bit count is not excessive, the reference 
quantization step size set for each first image unit is 
compared with the threshold quantization step size, for 
providing the quantization step size without any 

20 adjustment when the reference quantization step size is 
not exceeding the excess or shortage of the average bit 
rate, and adjusting the quantization step size according 
to the excess or shortage of the average bit count and 
selectively providing the greater one of the adjusted 

25 quantization step size and the threshold quantization 
step size, and when the excess or shortage of the average 
bit count is excessive, the quantization step size is 
adjusted according to the excess or shortage of the 
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average bit count, the adjusted quantizing with being 
provided as the quantizing with for each second image 
unit. 

According to other aspect of the present invention, 
5 there is provided a method of variable bit rate coding 
video data on the basis of a predetermined average bit 
rate wherein quantization step size is set such as to 
hold a constant coded image quality level over a plurality 
q of groups of picture images and the quantization step 

? 10 size is adjusted from the excess or shortage of the actual 
III generated code bit rate with respect to the average bit 

rate with reference to the quantization step size that 
ly has been set as above. 

^ In the present invention, the process of the 

1! 15 setting of the quantization step size by utilizing 

p features of image and the process of adjusting the average 

bit rate are realized separately for the variable bit 
rate coding. More specifically, the quantization step 
size is set such as to hold a constant coded image quality 
20 level over a plurality of groups of pictures. 

In the average bit rate adjusting process, the 
quantization step size is adjusted from the excess or 
shortage of the actual generated code bit rate with 
respect to the average bit rate with reference to the 
25 quantization step size that has been set as above. 

Consequently, it is possible to realize a variable bit 
rate coder with higher image quality than the prior art 
single path variable bit rate coder since the 
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quantization step size determined dependent on the scene 
becomes constant and then the quantization step size is 
adjusted so as to make the coding rate to the average 
bit rate. 

5 In addition, by providing a minimum quantization 

step size that can be set and also by controlling the 
adjustment of an excessive or insufficient code bit count 
according to the quantization step size determined in 
dependence on the scene and the excess or shortage of 

10 the code bit count with respect to the average bit rate, 
it is possible to suppress generation of excess codes 
and permit code consumption with images, which it is 
desired to spend more codes. Thus, it is possible to 
obtain more uniform image quality improvement, 

15 Other objects and features will be clarified from 

the following description with reference to attached 
drawings . 

BRIEF DE SCR IPTI ON OF THE DRAWIN GS 

Fig, 1 is a view for describing a method of variable 
20 bit rate video coding according to the present invention; 

Fig* 2 is a view for describing the quantization 
step size setting means 102 shown in Fig. 1; 

Fig. 3 is a view for describing the quantization 
step size adjusting means 105 shown in Fig. 1; 
25 Fig. 4 is a flow chart illustrating an example of 

operation according to the present invention; 

Fig. 5 is a block diagram showing the other 
embodiment of the present invention; 
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Fig. 6 is a block diagram showing the construction 
of the maximum bit rate adjusting means 104 shown in Fig, 
5; 

Fig, 7 is a flow chart illustrating the 
5 quantization step size updating process in the 
embodiment; and 

Fig. 8 is a block diagram of a prior art image coding 
method conforming to the MPEG-2 ( ISO/IEC-13818-2 ) . 

PREFERRED EMB OD IMENT S OF THE INVENTION 

10 Preferred embodiments of the present invention 

will now be described with reference to the drawings . 

Fig. 1 is a view for describing a method of variable 
bit rate video coding according to the present invention. 
It is assumed that an MPEG-2 ( ISO/ICE-13818 ) system is 

15 used as coding system. This coding system, however, is 
by no means limitative, and it is possible to use any 
coding technique so long as a scalar quantizing process 
is adopted and the bit count is controlled according to 
the quantization step size. For example, it is possible 

20 to use an MPEG-1 ( ISOIEC-11172 ) system, or an ITU-T H. 
261 or ITU-T H. 263 system. 

The quantizng width provided to the video coding 
means is used as a reference for bit count control, and 
it by no means interferes with adaptive quantization 

25 corresponding to a local position in the picture. This 
means that the finally provided adaptive quantization 
step size may be different from the quantization step 
size provided as reference or the bit count control to 
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the video coding means. 

It is further assumed that GOP (i.e., group of 
pictures) is a first image coding unit, and the 
macro-block is the second image coding unit. However, 
5 these units in the coding process are by no means 

limitative, and it is possible to use any units, i.e., 
a unit of image quality control for a predetermined period 
of time and a smaller unit for finely adjusting values 
obtained by this control. 

10 For example, the first image coding unit may be 

constituted by a plurality of GOPs, and the second image 
coding unit may be constituted by a plurality of 
macro-blocks. As further units in the coding process, 
an aggregation of macro-blocks in a row in the picture, 

15 a slice defined in MPEG-1 or MPEG-2 or GOB (i.e., Group 
Of Blocks) defined in H. 2 61 are conceivable. 

The function of the embodiment of the variable bit 
rate video coder shown in Fig. 1 according to the present 
invention will now be described. 

20 The variable bit rate video coder illustrated in 

Fig. 1 comprises a video coding means 101, a picture group 
quantization step size setting means 102 and an average 
bit rate adjusting means 103. The video coding means 101 
executes a coding process according to quantization step 

25 size Qj given for each macro-block, and as a result of 
the process it outputs generated bit count Sj„ 

The quantization step size Qj provided to the video 
coding means 101 is also provided to the quantization 
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step size setting means 102. The quantization step size 
setting means 102 sets reference quantization step size 
Qgop of GOP according to the quantization step size Qj 
thus provided and also the generated bit count Sj 
5 generated in the coding process, which is executed in 
the video coding means 101 according to the quantization 
step size Q j . 

The actually generated bit count S j provided to the 
quantization step size setting means 102 is also provided 

10 to the average bit rate adjusting means 103 . The average 
bit rate adjusting means 103 compares and obtains the 
difference between the actually generated bit count Sj 
and a average bit rate, and adjusts the reference 
quantization step size Qgop of GOP provided from the 

15 quantization step size setting means 102, thus 

determining quantization step size Qj provided to the 
video coding means 101. 

Fig. 2 is a view for describing the quantization 
step size setting means 102 shown in Fig. 1. Referring 

20 to the Figure, the quantization step size setting means 
102 includes a quantization step size accumulator 201, 
a generated bit count accumulator 202, a coding 
complaxity calculator 203 and a GOP quantization step 
size calculator 204. 

25 The quantization step size accumulator 201 

accumulates the quantization step size Qj, which is 
provided from the average bit rate adjusting means 103 
to the quantization step size setting means 102. The 
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accumulator 201 continues the operation of accumulating 
the quantization step size Qj over the entire coding 
process sequence. The generated bit count accumulator 
202 accumulates the generated bit count Sj, which is 
5 provided from the video coding means 101 to the 
quantization step size setting means 102. The 
accumulator 202 also continues the operation of 
accumulating the generated bit count over the entire 
sequence. 

5 10 The coding complexity calculator 203 calculates 

: d™ 

; J ! the complexity of each GOP from the average quantization 

l p step size Qj obtained from the summation of quantization 

II step size obtained in the quantization step size 

accumulator 201 and the summation of generated bit count 
ij 15 Sj obtained in the generated bit count accumulator 202. 

SSI 

The average quantization step size Qj is obtained from 
^ = the summation of quantization step size Qj provided from 

the quantization step size accumulator 2 01 and the number 
of times of output of the quantization step size Qj from 
20 the average bit rate adjusting means 103. The summation 
of generated bit count Sj is obtained as a result of 
accumulation of generated bit count Sj, which has been 
generated in the video coding means 101 according to the 
quantization step size Qj provided for each macro-block 
25 from the average bit rate adjusting means 103. 

The GOP quantization step size calculator 204 
computes reference quantization step size Ggop of each 
GOP from the GOP complexity of each GOP provided from 
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the GOP complexity calculator 203 and the average bit 
rate. 

Fig, 3 is a view for describing the quantization 
step size adjusting means 105 shown in Fig- 1. Referring 
5 to the Figure, the quantization step size ad justing means 
103 includes an average bit rate virtual buffer occupancy 
calculator 301 and a quantization step size modifier 302* 
The virtual buffer occupancy calculator 3 01 calculates 
a virtual buffer occupancy from the generated bit count 

10 Sj provided for each GOP from the video coding means 101 
and the average bit rate. 

The quantization step size modifier 302 modifies 
the quantization step size Qgop provided from the 
quantization step size setting means 102 according to 

15 the average bit rate virtual buffer occupancy calculated 
in the virtual buffer occupancy calculator 301, and sends 
out the modified quantization step size Qgop to the video 
coding means 101 and the quantization step size setting 
means 102. 

20 Fig. 4 is a flow chart illustrating an example of 

operation according to the present invention. The 
variable bit rate video coder is first initialized (step 
SI). In this step, the quantization step size setting 
means 102 is first initialized to provide initial 

25 reference quantization step size Qgop. Specifically, an 
initial quantization step size Qinit is preliminarily 
set. According to the initial quantization step size 
Qinit the setting means 102 is initialized to provide 
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the initialized reference quantization step size Qgop 
as 

Qgop = Qinit. 

Then, the accumulators, various counters and 
5 virtual buffer occupancy are reset. Specifically, in the 
quantization step size setting means 102, summation Ssum 
of the generated bit count S j generated in the generated 
bit count accumulator 201 and summation Qsum of the 
quantization step size Qj in the quantization step size 
10 accumulator 202 are reset to "0". Also, a macro-block 
number counter j and a GOP number counter Ngop are reset 
to "0". Furthermore, virtual buffer occupancy vboc is 
reset to "0". Mathematically, 
Ssum = 0 
15 Qsum = 0 

j = 0 
Ngop = 0 
and vboc = 0 . 

In subsequent step S2, the quantization step size 
20 adjusting means 103 updates the reference quantization 
step size Qgop provided from the quantization step size 
setting means 102. Specifically, the ad justing means 103 
executes adjustment of the reference quantization step 
size Qgop provided from the quantization step size 
25 setting means 102 according to the generated bit count 
Si generated in the coding process executed in the video 
coding means 101 and the buffer occupancy vboc obtained 
based on the average bit rate. 
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The quantization step size adjusting means 103 sets 
the adjusted quantization step size Qj as 

Qj = Qgop * (1 + vboc/Qreact) 
where Qreact is a parameter for determining the magnitude 
5 of reaction to control of the generated bit count Sj for 
making up for an excess or a shortage, according to the 
reference quantization step size Ggop set in the 
quantization step size setting means 102 before update 
and the virtual buffer occupancy vboc before update. 

10 In subsequent step S3, the video coding means 101 

executes a coding process on the next macro-block 
according to the quantization step size Qj provided from 
the quantization step size adjusting means 103. After 
the coding process which is executed on one macro-block, 

15 for instance, the quantization step size setting means 
102 executes step S4 of updating the virtual buffer 
occupancy vboc and the summation Ssum of the generated 
bit count set therein. To update the virtual buffer 
occupancy vboc, the setting means 102 adds the generated 

20 bit count Sj generated in the coding process on a j- 
th macro-block to vboc and subtracts the average bit rate 
Rave_mb from the sum. 

In addition to the updating of the summation Ssum 
of the generated bit count set in the generated bit count 

25 accumulator 202, the coding means 102 also updates the 
summation Qsum of the bit count set in the quantization 
step size accumulator 2 01, and further increments a 
macro-block number counter j. Mathematically, 
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vboc 



vboc + Sj - Rave_mb 



Ssum 



Ssum + Sj 



Qsum 



Qsum + Qj 



and 



j + 1 



r 10 



m 15 
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As is seen from the above calculations, the virtual 
buffer occupancy represents a bit balance of the actually 
generated bit count to the average bit rate Rave_mb. 

In subsequent step S5, a judgment is made as to 
whether the macro-block to be coded is the last 
macro-block in GOP * When it is detected as a result of 
this judgment that the macro-block to be coded is other 
than the last macro-block of GOP (NO in step S5), the 
routine goes back to the step S2 to execute the coding 
process repeatedly on the next macro-block. 

When it is detected in the judgment step S5 that 
the macro-block to be coded is the last macro-block of 
GOP (YES in step S5), a GOP complexity updating process 
is executed. Specifically, at this time a coding process 
on the next GOP is executed. At this time, the GOP number 
counter Ngop is incremented. Also, the GOP complexity 
calculator 203 in the quantization step size setting 
means 102 calculates new GOP complexity Xgop for updating 
from the summation Ssum of the generated bit count and 
the summation Qsum of the generated bit count in the 
entire sequence of the coding process on the preceding 
GOP (step S6). Mathematically, 

Ngop = Ngop + 1 
and Xgop = Ssum * (Qsum/ j ) /Ngop 
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Furthermore, the GOP quantization step size 
calculator 204 updates the reference quantization step 
size Qgop of the preceding GOP (step S7) . Specifically, 
the calculator 204 calculates new reference quantization 
5 step size Qgop for updating from the ratio of the GOP 
complexity obtained as the above equation to the average 
bit rate Rgop per GOP • Mathematically, 

Qgop = Xgop/Rgop 

In subsequent step S8, a judgment is executed as 

10 to whether the coding process on the last macro-block 
of GOP has been ended. When it is detected in the judgment 
that the coding process on the last macro-block has been 
ended (YES in step S8), an end is brought to the coding 
process on this GOP. When it is detected in the judgment 

15 step S8 that the macro-block having just been coded is 
other than the last macro-block of GOP, i.e., that the 
coding process on this GOP has not yet been ended (NO 
in step S8) , the routine goes back to the coding process 
step S2 for executing the coding process on the succeeding 

20 macro-block. 

A different embodiment of the present invention 
will now be described with reference to Figs. 5 to 7. 

Fig. 5 is a block diagram showing the other 
embodiment of the present invention. This embodiment is 

25 obtained by adding a maximum bit rate adjusting means 
104 to the previous embodiment shown in Fig. 1. The 
maximum bit rate adjusting means 104 controls the 
quantization step size according to the quantization 
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step size Qa outputted from the average bit rate adjusting 
means 103 and generates the bit count Sj outputted from 
the video coding m€>ans 101 such as to observe the 
restriction on the maximum bit rate set therein, and 
5 output quantization step size Qj in second image units. 

Fig* 6 is a block diagram showing the construction 
of the maximum bit rate adjusting means 104 shown in Fig. 
5 . Referring to the Figure, the maximum bit rate adjusting 
means 104 includes a constant bit rate control means 401 

10 and a quantization step size selector 402. 

The constant bit rate control means 401 calculates 
the quantization step size in macro-block units by 
executing constant bit rate coding at a predetermined 
maximum bit rate. The quantization step size selector 

15 402 compares the quantization step size calculated in 
the constant bit rate control means 401 and the 
quantization step size Qa set in the average bit rate 
adjusting means 103 by average bit rate control up to 
the immediately preceding coded macro-block, and selects 

20 the larger one as quantization step size Qj to be provided 
to the video coding means 101. 

The constant bit rate control method may adopt, for 
instance, a "Test Model 5 method". 

As a modification of the embodiment, a means for 

25 controlling the quantization step size outputted from 
the average bit rate adjusting means may be used, in which 
a minimum quantization step size Qmin is preliminarily 
set. In this case, the larger one of the preliminarily 



28 



set minimum quantization step size Qmin and the 
quantization step size Qa outputted from the average bit 
rate adjusting means 103 may be provided as quantization 
step size Qj given as 
5 Qj - max(Qa, Qmin) 

to the video coding means 101 and the quantization step 
size setting means 102. 

As a different modification of the embodiment, a 
means for controlling the quantization step size 
10 outputted from the average bit rate adjusting means may 
be used, which sets a maximum excess bit count Dmax from 
the average bit rate and controls the quantization step 
size by using the bit count Dmax and the virtual buffer 
occupancy vboc. For example, as for parameter Qreact used 
15 for the adjustment of the quantization step size 
according to the virtual buffer occupancy vboc, 
utilizable bit count (Dmax-vboc) may be calculated by 
subtracting vboc from the maximum excess bit count Dmax, 
and the smaller one of Qreact and (Dmax-vboc) may be used 
20 as the parameter Qreact, i.e., 

Qreact = min( Qreact, Dmax-vboc). 
Alternatively, it is possible to use 
Qj = Qgop* ( 1+vboc-vboc/ (Dmax-vboc) 
or Qj = Qgop*/ (Dmax-vboc ) 
25 instead of the parameter Qreact. 

As a further modification of the embodiment, in the 
coding process on each GOP, the GOP complexity X of each 
GOP may be computed from the average quantization step 
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size obtained from the summation Ssum.gop of the bit count 
Sj generated in the video coding means 101 and the 
summation Qsum.gop of the quantization step size Qj 
outputted per GOP from the average bit rate adjusting 

5 means 103 and the product of these parameters , and average 
GOP complexity Xgop may be obtained over these sequences, 
i.e., the coding processes on individual GOPs. 

Specif ically, when a coding process on an i-th GOP 
is ended the summation Ssum.gop of the bit count and the 

10 summation Qsum.gop of the quantization step size are 
obtained for the i-th GOP, and the GOP complexity X(i) 
of the i-th GOP is obtained from the product of these 
parameters. The summation of the quantization step size 
is set in macro-block units, and Nmb.gop is the number 

15 of macro-blocks per GOP. 

In this case, assuming the first GOP serial number 
is "1", Xgop is calculated as 

X(i) = Ssum.gop* (Qsum.gop /Nmb.gop) 
and Xgop = (X(i)+X( i-1 )+ ... + X(l))/i 

20 The average GOP complexity Xgop may not be over the 

GOPs of GOP serial numbers 1 to i, but it may be over 
any previously coded GOPs, that is, in a further 
modification of the embodiment it may be 

Xgop = (X(i)+X(i-1)+ ... +X(i-N) )/(N-l) 

25 where N is a parameter for setting the number of GOPs 
for the average computation. 

As a further modification of the embodiment, the 
reference quantization step size Qgop may be set from 



30 



the relation between the average GOP complexity Xave, 
which is computed over the entire sequence of calculation 
of the GOP complexity X of each GOP, i.e. , over the GOPs 
of GOP serial numbers 1 to i, and the average GOP 

5 complexity Xgop, which is calculated over GOPs from the 
immediately preceding coded GOP to GOP of GOP serial 
number i . 

For example, mathematically 

xave = (X(i)+X(i-1) + ... + X(l))/i 

10 Xgop = (X)i)+X(i-1) + ... + X(i-N))/(N+1) 

and Qgop = (Xave/Rgop) * ( 1+a* (Xgop-Xave) ) 
where a is a parameter for determining the strength of 
control when setting Ggop according to the difference 
of X obtained from (Xgop-Xave). The parameter a may be 

15 set as 

a = a'/5x 

As a further modification of the embodiment, a 
threshold Qth as an upper limit quantization step size, 
may be provided for average bit rate adjustment when the 
20 reference quantization step size Qgop is adjusted in the 
average bit rate adjusting means 103. For example, the 
step S2 of the quantization step size updating process 
as described before in connection with the flow chart 
of Fig. 2, may be executed as shown in the flow chart 

25 of Fig. 7. 

First, the average bit rate adjusting means 103 
checks whether the bit count Sj generated in the coding 
process on each macro-block is in excess of the average 
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bit rate assigned per macro-block (step Sll). When 
detecting in the check that the bit count S j is in excess 
of the average bit rate (YES in step Sll), the average 
bit rate adjusting means 103 updates the reference 

5 quantization step size Qgop inputted from the 

quantization step size setting means 102 (step S12) • The 
updated quantization step size Qj is outputted to the 
video coding means 101 and also to the quantization step 
size setting means 102. 

10 When detecting in the check step Sll that the bit 

count Sj is not in excess of the average bit rate (NO 
in step Sll), the average bit rate adjusting means 103 
checks whether the reference quantization step size Qgop 
inputted from the quantization step size setting means 

15 102 is exceeding a threshold Qth, i.e., upper limit, of 
the quantization step size (step S13). When detecting 
in this check that the reference quantization step size 
Qgop inputted from the quantization step size setting 
means 102 is above the threshold quantization step size 

20 Qth (YES in step S13), like the step S12 noted above, 
the average bit rate adjusting means 103 updates the 
reference quantization step size Qgop inputted from the 
GOP quantizing with setting means 102 (step S14). The 
updated quantization step size Qj is compared with the 

25 threshold Qth, and the greater one is selected ( step S15 ) . 

Detecting in the check step S13 that the reference 
quantization step size Qgop inputted from the 
quantization step size setting means 102 is below the 
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threshold quantization step size Qth (NO in step S13), 
the average bit rate adjusting means 103 does not update 
the reference quantization step size Qgop inputted from 
the quantization step size setting means 102. In this 

5 case, the reference quantization step size Qgop is 
outputted as Qj to the video coding means 101* 

As has been shown, the average bit rate adjusting 
means 103 does not always execute adjustment of the 
reference quantization step size inputted from the 

10 quantization step size setting means 102, but it executes 
the adjustment process in dependence on whether the bit 
count of codes generated in the coding process on each 
macro-block is excessive or insufficient relative to the 
average bit rate assigned per macro-block and also the 

15 magnitude relation between the reference quantization 
step size and the threshold or upper limit quantization 
step size. 

Thus, when the reference quantization step size 
inputted from the quantization step size setting means 

20 102 is below the upper limit quantization step size and 
the bit count generated in the coding process per 
macro-block is below the average bit rate assigned per 
macro-block, it is possible to suppress the bit count 
of wasteful codes generated for making up for an excess 

25 of codes generated at the average bit rate. 

Thus, when an image requires many bit count, it is 
possible to spend the suppressed bits, thus improving 
the average image quality. 
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As has been described in the foregoing , according 
to the present invention it is possible to execute real 
time processing for variable bit rate coding and also 
higher image quality coding than obtainable with the 
5 prior art one-path variable bit rate coder, which 

executes the real time processing for variable bit rate 
coding. 

Changes in construction will occur to those skilled 
in the art and various apparently different 

10 modifications and embodiments may be made without 

departing from the scope of the present invention. The 
matter set forth in the foregoing description and 
accompanying drawings is offered by way of illustration 
only. It is therefore intended that the foregoing 

15 description be regarded as illustrative rather than 
limiting. 
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What is claimed is: 

1 . An apparatus for variable bit rate video coding 
on the basis of a predetermined average bit rate 
comprising: 

a video coding means for coding input video with a 
predetermined quantization step size and providing coded 
data and a generated code bit count; 

a quantization step size setting means for setting 
a reference quantization step size for each first image 
unit in correspondence to the average bit rate from the 
quantization step size provided to the video coding means 
and also the generated bit count provided therefrom; and 

a quantization step size adjusting means for 
adjusting the quantization step size provided from the 
quantization step size setting means for each second image 
unit from the generated bit count provided from the video 
coding means and also from an excess or a shortage of the 
average bit rate. 

2 . The apparatus for variable bit rate video coding 
according to claim 1, wherein: 

the first image unit is a group of a plurality of 
picture frames . 

3 . The apparatus for variable bit rate video coding 
according to claim 1/ wherein: 

the second image unit is a group of a plurality of 
macro-blocks as divisions of one picture frame. 
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4 . The apparatus for variable bit rate video coding 
according to claim 1, wherein: 

the quantization step size setting means for setting 
the reference quantization step size for each first image 
unit computes an image unit complexity defined by the 
product of the average quantization step size over the 
preceding coded image and generated bit count, and also 
computes the reference quantization step size for each 
first image unit from the ratio between the first image 
unit complexity and the average bit rate. 

5 . The apparatus for variable bit rate video coding 
according to claim 1, wherein : 

the quantization step size adjusting means for 
adjusting the quantization step size for each second image 
unit sets a minimum permissible quantization step size. 

6 . The apparatus for variable bit rate video coding 
according to claim 1, wherein: 

the quantization step size adjusting means for 
adjusting the quantization step size for each second image 
unit sets a maximum excess of the average bit rate, sets 
a virtual buffer sized to the maximum excess from which 
codes are withdrawn at the average bit rate and adjusts 
the quantization step size for each second image unit by 
using the buffer occupancy in the virtual buffer and a 
utilizable buffer size obtained by subtracting the buffer 
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occupancy in the virtual buffer from the size thereof. 

7 . The apparatus for variable bit rate video coding 
according to claim 1, wherein: 

the quantization step size setting means for setting 
the reference quantization step size for each second image 
unit includes a means for computing a first quantization 
step size by adjusting the quantization step size for each 
second image unit from the generated bit count provided 
from the video coding means and the excess or shortage of 
the average bit rate; and 

it sets a maximum bit rate, computes a second 
quantization step size, which is set in the case of 
constant bit rate control on the basis of the maximum bit 
rate, from the quantization step size set in the 
quantization step size setting means and the generated bit 
count provided from the video coding means, and provides 
the greater one of the first and second quantization step 
sizes to the video coding means. 

8 . The apparatus for variable bit rate video coding 
according to claim 1, wherein: 

the quantization step size setting means for setting 
the reference quantization step size for each first image 
unit computes a first image unit complexity for each first 
image unit, the first image unit complexity being defined 
by the product of the summation of generated bit count and 
the average quantization step size for each first image 
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unit, and determines the quantization step size of the 
first image unit from the first image unit complexity of 
the first image unit and the bit count per first image unit, 

9 • The apparatus for variable bit rate video coding 
according to claim 1, wherein: 

the quantization step size setting means for setting 
the reference quantization step size for each first image 
unit computes the first image unit complexity for each 
first image unit from the summation of the generated bit 
count and the average quantization step size for each first 
image unit, and determines the quantization step size of 
the first image unit from the average first image unit 
complexity for the first image unit over the preceding 
coded image , the average first image unit complexity of 
the immediately preceding first image unit or a plurality 
of first image unit including the immediately preceding 
first image unit and the bit rate per first image unit. 

10. The apparatus for variable bit rate coding video 
data according to claim 1, wherein: 

the quantization step size adjusting means for 
adjusting the quantization step size for each second image 
unit preliminarily sets a threshold for quantization step 
size, 

when the excess or shortage of the average bit rate 
is not excessive, the reference quantization step size set 
for each first image unit is compared with the threshold 
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for quantization step size, for 

providing the quantization step size without any 
adjustment when the reference quantization step size is 
not exceeding the threshold quantization step size, and 

adjusting the quantization step size according to 
the excess or shortage of the average bit rate and 
selectively providing the greater one of the adjusted 
quantization step size and the threshold for quantization 

step size, and 

when the excess or shortage of the average bit rate 
is excessive, the quantization step size is adjusted 
according to the excess or shortage of the average bit rate, 
the adjusted quantization step size being provided as the 
quantization step size for each second image unit* 

11. A method of variable bit rate coding video data 
on the basis of a predetermined average bit rate 
comprising: 

a video coding step of coding input video with a 
predetermined quantization step size and providing coded 
data and a generated code bit count; 

a quantization step size setting step of setting a 
reference quantization step size for each first image unit 
in correspondence to the average bit rate from the 
quantization step size provided to the video coding step 
and also the generated code bit count provided therefrom; 
and 

a quantization step size adjusting means for 
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adjusting the quantization step size provided from the 
quantization step size setting means for each second image 
unit from the generated code bit count provided from the 
video coding step and also from an excess or a shortage 
of the average bit rate. 

12 . The method of variable bit rate coding video data 
according to claim 11/ wherein: 

the first image unit is a group of a plurality of 
picture frames. 

13 . The method of variable bit rate coding video data 
according to claim 11 , wherein: 

the second image is a group of a plurality of 
macro-blocks as divisions of one picture frame. 

14 . The method of variable bit rate coding video data 
according to claim 11, wherein: 

in the quantization step size setting step of 
setting the reference quantization step size for each 
first image unit, a first image unit complexity is computed, 
which is defined by the product of the average quantization 
step size and generated code bit rate over the preceding 
coded image, and also the reference quantization step size 
for each first image unit is computed from the ratio 
between the first image unit complexity and the average 
bit rate. 
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15 . The method of variable bit rate coding video data 
according to claim 11, wherein: 

in the quantization step size adjusting step of 
adjusting the quantization step size for each second image 
unit, a minimum permissible quantization step size is set. 

16. The method of variable bit state coding video 
data according to claim 11, wherein: 

in the quantization step size adjusting step of 
adjusting the quantization step size for each second image 
unit, a maximum excess of the average bit rate is set, a 
virtual buffer sized to the maximum excess is set for 
withdrawing codes at the average code bit, and the 
quantization step size is adjusted for each second image 
unit by using the occupation content in the virtual buffer 
and a utilizable code bit count obtained by subtracting 
the occupation content in the virtual buffer from the size 
thereof. 

17 . The method of variable bit rate coding video data 
according to claims 11, wherein: 

the quantization step size setting step of setting 
the reference quantization step size for each second image 
unit includes a step of computing a first quantization step 
size by adjusting the quantization step size for each 
second image unit from the generated code bit count 
provided from the video coding step and the excess or 
shortage of the average bit rate; and 
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in the computing step, a maximum bit rate is set, 
a second quantization step size is computed, which is set 
in the case of fixed bit rate control on the basis of the 
maximum bit rate, from the quantized width set in the 
quantization step size setting step and the generated code 
bit count provided from the video coding step, and the 
greater one of the first and second quantization step sizes 
is provided to the video coding step. 

18. The method of variable bit rate coding video 
according to claim 11, wherein: 

in the quantization step size setting means for 
setting the reference quantization step size for each 
first image unit, a first image unit complexity is computed 
for each first image unit, the first image unit being 
defined by the product of the summation of generated code 
bit rate and the average quantization step size for each 
first image unit, and the quantization step size of the 
first image is determined from the first image unit 
complexity of the first image unit and the bit rate per 
first image unit. 

19. The method of variable bit rate coding video 
according to claims 11, wherein: 

in the quantization step size setting means for 
setting the reference quantization step size for each 
first image unit, the first image unit complexity is 
computed for each first image unit from the summation of 
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the generated code bit count and the average quantization 
step size for each first image unit, and the quantization 
step size of the first image unit is determined from the 
average first image unit complexity for the first image 
unit over the preceding coded image, the average first 
image unit complexity of the immediately preceding first 
image unit or a plurality of first image unit including 
the immediately preceding first image unit and the bit rate 
per first image unit. 

20. The method of variable bit rate coding video 
coding according to claims 11, wherein: 

in the quantization step size adjusting means of 
adjusting the quantization step size for each second image 
unit, a threshold quantization step size is preliminarily 
set, when the access or shortage of the average bit count 
is not excessive, the reference quantization step size set 
for each first image unit is compared with the threshold 
quantization step size, for 

providing the quantization step size without any 
adjustment when the reference quantization step size is 
not exceeding the excess or shortage of the average bit 
rate, and 

adjusting the quantization step size according to 
the excess or shortage of the average bit count and 
selectively providing the greater one of the adjusted 
quantization step size and the threshold quantization step 
size, and 
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when the excess or shortage of the average bit count 
is excessive, the quantization step size is adjusted 
according to the excess or shortage of the average bit 
count, the adjusted quantizing with being provided as the 
quantizing with for each second image unit. 

21. A method of variable bit rate coding video data 
on the basis of a predetermined average bit rate wherein 
quantization step size is set such as to hold a constant 
coded image quality level over a plurality of groups of 
picture images and the quantization step size is adjusted 
from the excess or shortage of the actual generated code 
bit rate with respect to the average bit rate with 
reference to the quantization step size that has been set 
as above • 
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The quantization step size is set on the basis of 
features of image and the average bit rate is adjusted 
for the variable bit rate coding. In more specif ically, 

5 the quantization step size is set such as to hold a 
constant coded image quality level over a plurality of 
groups of pictures. The quantization step size is 
adjusted from the excess or shortage of the actual 
generated bit count with respect to the average bit rate 

10 with reference to the quantization step size that has 
been set as above. 
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